In recent years, research and development of augmented reality technology has been actively conducted in which virtual images are projected onto real objects using a projector or the like in order to present notes, menus and the like associated with the real objects.
Meanwhile, in recent years, in order to improve usability of information terminal devices such as personal computers and smartphones, research and development of natural user interfaces that enable operations by actions that are more natural for humans and that are more intuitive have been actively conducted. For example, screen operations and voice conversations using hand gesture actions have been put into practical use and have begun to be utilized as new user interfaces in place of the keyboard, mouse and touch panel.
In addition, as an attempt to realize a more realistic user interface, use of actions such as touching and moving a real object have also been proposed. For example, a user interface technique has been realized in which a displayed object displayed by projection using a projector or the like is handled as if it were being directly controlled using fingers and other tools.
In techniques such as the user interface technique in which a displayed object is treated as if it were being directly controlled by a finger, the finger is recognized by detecting the area of the finger (hereinafter referred to as a finger area) from an image captured by an imaging device. As a technique for detecting the finger area, for example, a method is known in which a skin color area is extracted from a captured image, and a finger area is detected based on the features of the extracted shape. In this method, the finger area is detected by converting the captured image into an HSV (Hue, Saturation, Value) color representation image (hereinafter referred to as an HSV image) and performing skin color extraction and a binarization process.
In the skin color extraction, pixels included in the skin color area defined in the HSV color representation are extracted as the skin color. However, when nails are decorated with coloring, such as artificial nails and the like, the color distribution of the nail area may be excluded from the skin color area. This makes it difficult to accurately detect the finger area including the nail tip. Hereinafter, a nail decorated with coloring such as an artificial nail or the like is referred to as a decorated nail.
In the user interface technique in which a displayed object is treated as if it were being directly controlled by a finger, accurate detection of the fingertip position is very important because the displayed object is controlled according to the fingertip position.
In this regard, a method has been proposed for adjusting the color area that is to be detected as a finger area, so as to include the color distribution of a decorated nail. However, in the method in which the color area is adjusted, it is not always possible to detect the finger area including the nail tip, because nail decorations are in various colors. In addition, the color area is expanded so as to include the color distribution of decorated nails, and this increases the possibility of erroneously detecting the background as the finger area.
Meanwhile, techniques described in each of the following documents have been known.    Japanese Laid-open Patent Publication No. 2003-346162    Japanese Laid-open Patent Publication No. 2002-209082    Japanese Laid-open Patent Publication No. 2009-265809